User Behavior Recognition for an Automatic Prompting System - A Structured Approach based on Task Analysis

In this paper, we describe a structured approach for user behavior recognition in an automatic prompting system that assists users with cognitive disabilities in the task of brushing their teeth. We analyze the brushing task using qualitative data analysis. The results are a hierarchical decomposition of the task and the identification of environmental configurations during subtasks. We develop a hierarchical recognition framework based on the results of task analysis: We extract a set of features from multimodal sensors which are discretized into the environmental configuration in terms of states of objects involved in the brushing task. We classify subtasks using a Bayesian Network (BN) classifier and a Bayesian Filtering approach. We compare three variants of the BN using different observation models (IU, NaiveBayes and Holistic) with a maximum-margin classifier (multi-class SVM). We present recognition results on 18 trials with regular users and found the BN with a NaiveBayes observation model to produce the best recognition rates of 84.5% on avg.